A note on “infinitely often,” “probability 1,” and the law of large numbers

These notes give details on the probability concepts of “infinitely often” and “with probability 1.” This is useful for understanding the Borel-Cantelli lemma and the strong law of large numbers. I. SEQUENCES OF EVENTS A. Events Consider a general sample space S. Recall that an event is a subset of S that has a well defined probability. That is, a set A is an event if and only if A ⊆ S and P [A] exists. Formally, a probability experiment introduces both a sample space S and a sigma algebra F . The sigma algebra F contains all events, that is, it is the collection of all subsets of S for which probabilities are defined. B. Can an event be “true”? It is common to talk about events as being either “true” or “false.” However, if an event is just a subset of S , does it make sense to say that an event can be “true”? When we say that an event A is “true” (or that an event A “occurs”), we are imagining a probability experiment that produces an outcome ω ∈ S for which ω ∈ A. The event is “false” if the outcome satisfies ω / ∈ A. C. Shrinking a sequence of events Let {An}n=1 be an infinite sequence of events. Let A be another event. We say that An ↘ A if: • An ⊇ An+1 for all n ∈ {1, 2, 3, . . .}. • ∩n=1An = A We know that if An ↘ A then P [An]↘ P [A]. D. Infinitely often Let {An}n=1 be an infinite sequence of events. We say that events in the sequence occur “infinitely often” if Ai holds true for an infinite number of indices i ∈ {1, 2, 3, . . .}. Define {Ai i.o} as the event that an infinite number of events Ai occur. Formally, this new event needs to be a subset of S. Hence: {Ai i.o} = {ω ∈ S : ω ∈ Ai for an infinite number of indices i ∈ {1, 2, 3, . . .}} (1) Lemma 1. For any sequence of events {An}n=1 we have: ∪i=n Ai ↘ {Ai i.o} (2) and so lim n→∞ P [∪i=nAi] = P [Ai i.o.] Proof. To prove (2), note that: ∪i=nAi ⊇ ∪i=n+1Ai ∀n ∈ {1, 2, 3, . . .} It remains to show that: ∩n=1 ∪i=nAi = {Ai i.o.} (3) This can be done by considering two cases (left as an exercise): 1) Case 1: Suppose {Ai i.o} is true. Show that ∪i=nAi must be true for all n ∈ {1, 2, 3, . . .}. 2) Case 2: Suppose {Ai i.o} is false. Show that ∪i=nAi must be false for some n ∈ {1, 2, 3, . . .}. Note that equation (3) in the above proof shows that the event {Ai i.o.} can be written in terms of unions and intersections of the original events Ai. Formally, this means that the set {Ai i.o.} defined in (1) is indeed in the sigma algebra of events. UNIVERSITY OF SOUTHERN CALIFORNIA, SPRING 2016 2 E. Finitely often We say that Ai occurs “finitely often” if Ai holds true for an at most finite number of indices i ∈ {1, 2, 3, . . .}. Specifically: {Ai f.o.} = {Ai i.o.} By taking complements in Lemma 1 we obtain: ∩i=nAi ↗ {Ai f.o.} =⇒ lim n→∞ P [∩i=nAi ] = P [Ai f.o.] Taking complements of (3) gives: ∪n=1 ∩i=n Ai = {Ai f.o.} F. Preliminaries for Borel-Cantelli lemma We need two facts about real numbers: 1) log(1 + x) ≤ x for all x ∈ (−1,∞). 2) If {xi}i=1 is a sequence of non-negative real numbers such that ∑∞ i=1 xi <∞, then limn→∞ ∑∞ i=n xi = 0. The second fact is proven as follows: For all positive integers n we have: ∞ ∑